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Abstract. We present an analytically solvable random graph model in which the 
connections between the nodes can evolve in time, adiabatically slowly compared to 
the dynamics of the nodes. We apply the formalism to finite connectivity attractor 
neural network (Hopfield) models and we show that due to the minimisation of 
the frustration effects the retrieval region of the phase diagram can be significantly 
enlarged. Moreover, the fraction of misaligned spins is reduced by this effect, and is 
smaller than in the infinite connectivity regime. The main cause of this difference 
is found to be the non-zero fraction of sites with vanishing local field when the 
connectivity is finite. 



1. Introduction 

Random graphs play an important role in a wide range of scientific disciplines. The 
topological properties of graphs offer, in the language of nodes and edges, a simple, 
yet powerful representation of many different complex systems. For instance in 
biophysics, simple models of random graphs have been used to predict the function 
of a chemical component (a node or a 'spin' on the graph) given information about how 
different chemicals interact with one another (graph connectivity) and their individual 
functionality within the cell (spin orientation). In complex systems such as financial 
markets, one wishes to understand the underlying laws and critical parameter values 
relating to economic recession or success, given a certain number of connections between 
traders and their individual decisions. Examples of such complex systems are in 
abundance, ranging from the aforementioned to ecological and linguistic networks (for 
recent reviews on the subject see P3I2])- These systems share one common feature: the 
sparse connectivity between the nodes. Every member of the population is connected 
on average to only a small fraction of all other members; yet a collective behaviour is 
observed. 

Under this unifying framework, random graphs stand as an interesting entity per se. 
From a statistical mechanical point of view such systems differ fundamentally from fully- 
connected ones, especially in the parameter regime of finite connectivity. In the latter 
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case, even at the level of replica symmetry, one finds that an order-parameter function is 
required to describe equilibrium j3llll21ini- Several applications of analytically solvable 
models on finite connectivity random graphs can be found for e.g. error-correcting codes 
13 E], cryptography CHI, optimisation problems [TTJ [T2], spin glasses [121 E] and 
neural networks [121 Uni HI] ■ 

Physical systems however hardly ever maintain a static network architecture: the 
structure of web-links changes continuously over time, as does the connectivity between 
traders in stock markets, or the proteinic interactions through the evolutionary process; 
dynamically evolving graphs form the majority of complex systems in nature. Studying 
such systems from an analytic viewpoint poses an important theoretical challenge, since 
very little is known about the dynamics of disordered systems with finite connectivity. 
In this paper, we solve a simple model of a spin system on a random graph in which 
the network architecture can evolve in time, although on timescales adiabatically slow 
compared to the dynamics of the spins. Thus spins are always at equilibrium with 
respect to the dynamics of the connections. This important class of models which 
can be solved exactly has been first studied for fully-connected systems and Hopfield 
models [IHl Hni I2II1 1211 1221 12H1 121| , whereas, curiously, the extension of these ideas to 
spin systems evolving on a hierarchy of timescales reproduces the Parisi full replica 
symmetry breaking scheme 

Here we extend these ideas to finite-connectivity systems and, in particular, we 
apply our formalism to Hopfield models. We show that the retrieval region of the phase 
diagram becomes significantly enlarged, and in fact, any finite degree of mobility of the 
graph can lead to recall of a condensed pattern at any finite value of the storage ratio. 
To get a better understanding of this effect, we also compute the fraction of misaligned 
spins and the fraction of sites with vanishing local fields as functions of the ratio of 
the characteristic temperatures of the two dynamic processes. In the limit of extreme 
dilution our expressions reproduce the results of pH] . 

2. Model definitions 

The present model is an extension of the one presented in [21] where geometry was of 
an adaptive nature. There, one could make important analytic simplifications owing to 
the choice of 'extreme dilution' scaling whereby each node in the graph was connected 
on average to a vanishing fraction of other nodes although this fraction still contained 
an infinite number of nodes. Here, we extend the formalism of [21] to consider the more 
realistic (albeit analytically more involved) scenario in which for every node there is 
a finite average number of connections, prescribed to a fixed number c. Thus, loops 
although rare are still present. 

To be precise, our model describes a graph of i = 1, . . . , nodes associated with 
(fast) neuron variables cr = (cri, . . . , o"Ar) with cTj G { — 1, 1}. For every pair of neurons 
we consider the connectivity variables c = {cy} with Cji = Cij and Cu = 0, which 
describe whether a connection between neurons cTj and aj is present {cij = 1) or not 
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{cij = 0). We take neuron variables to evolve according to a Glauber dynamics and at 
equilibrium their energy is described by 

H,{a,c) = -j:'-^±^nfa,a, (1) 



i<j ^ IM= 



The (quenched) variables {^^} € {—1, 1}^ with fi = 1, . . . ,p describe p binary patterns 
which have been imposed as attractors of the dynamics (stored Hopfield memories). The 
value of the pattern interactions J^fj. ^i^j remains unchanged throughout the dynamics. 
Neuron variables equilibrate with respect to this Hamiltonian at temperature and 
are described by the partition function 

Zf(c) = J2 e-^^f^*^'") (2) 

Within the timescale in which neurons evolve to their equilibrium state the connectivity 
variables Cij are effectively quenched. On larger timescales though, these also evolve 
dynamically (according to some Glauber prescription and preserving detailed balance) 
with the total Hamiltonian 

H^{c) = -^logZf(c) + ilog (^) E% (3) 

P p ^ ^ i<j 

By this construction, energetically favourable configurations of the connectivity matrix 
c are taken to be those that minimise the free energy of the neuron (fast) variables. 
The chemical potential in Q ensures, as was shown in |2n|! that on average there are 
c connections per neuron. Sparse connectivity then requires taking the limit c/N ^ 
while our choice of finite connectivity scaling corresponds to c ~ 

The variables c, in turn, equilibrate at an inverse temperature (3 with a total 
partition function given by 

= J^e-'^^^^^) = [Zf(c)]'^/^e-'°^(^)^«<. '=- (4) 

{c} {c} 

This partition sum effectively contains n = (3 / (3 replicated copies of the fast system, 
producing a replica theory with nonvanishing but, generally, noninteger replica- 
dimension. Hence, by definition, a large replica dimension n corresponds to low 
temperatures T (or, equivalently, low-energies) of the slow system relative to that of 
the fast. The formation of a particular graph configuration is then dominated by the 
Hamiltonian (jS)) rather than the thermal noise T. The total construction of equations 
(^-(^ therefore rearranges the geometry of the graph into optimised configurations c 
such that the retrieval performance of pattern recall can be potentially enhanced (due 
to the minimisation of frustrated bonds). 

At total equilibrium we will be interested in the evaluation of the (slow) free energy 

^ = -/J^"' (5) 
which generates expressions for the system's macroscopic observables. 
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3. Calculation of the RS free energy 



To evaluate the partition function first the trace over the connectivity variables is 
taken. This results in 



E n 



1 + ^e^L^,(^^-(^. 



(6) 



where we introduced the notation cr = (cr^,...,a") with cr ■ cr' = cr°cr'° and 
■ = I]^=i ^i^j as usual. The methodology required to proceed further from the 
above equation depends on the scaling regime one considers. For systems with extreme- 
dilution where c — oo (while c/N — > 0) [2^1 one can expand the above exponential 
and retain only the lowest two moments, which for ^ oo describe the system's 
thermodynamics fully. In contrast, for any finite c moments of order higher than 2 do 
not vanish. One is therefore required to consider an order-parameter function. 

At this stage, it turns out helpful to partition our graph into 2^ sublattices [131211, 
= = with sublattice-averages (J-'(^))^ = J2^P^ ^{^)^ where = \I^\/N. 
Then, upon introducing into our equations the order-function 



^^1 lei, 



(7) 



we obtain the following extremisation problem for 



/ = Extr{p^(cr)} 



c 

'2P 



\(T(T' 



The order-function -P^(cr) follows from the self-consistent equation 



+ 5 (logE^^P 
P \ cr 



(8) 



exp 




e^ei'^-^')^; 




(Eo-" exp 


c{EcT'P^i< 









(9) 



3.1. Replica- and sublattice- symmetric assumptions 

Our expressions ©, which describe the underlying order- function of our system, depend 
on the unpleasant sublattice index ^. Therefore solving this system of 2^ equations will 
become increasingly hard with the number of patterns p. However, if the system is in a 
state where there is a finite overlap only with one pattern (say ^ = 1), we expect that one 
can make a 'condensed ansatz', namely requiring the sublattice distribution of replicated 
spins to depend only on the component of the sublattice vector ^ corresponding to the 
condensed pattern, i.e., 

pact) = P^.ia) (10) 
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Taking the traces over the non-condensed pattern components then reduces the order 
parameter equations Q to :j: 

P^ia) ~ exp |^X:Kh(^^ ■ T)]^-iX:n'(^)e^^'^^'^-^ j (11) 

By inspection of the RHS of one concludes that the dependence on the remaining 
sublattice ^ can only come in in the form -P^(cr) = P(^cr). Upon inserting this form into 
the RHS of (fTTj) . it follows that the resulting equation is self consistent, as it should be: 

_ explcEr^T) ef(^-^) [cosh(f ^ ■ t)]^-^} 

E cr' exp {cEt ^(-r) e^^'^' '^^[cosh(fcr' ■ t)]p-i} 

This expression, devoid completely of pattern variables, allows for a significant reduction 
of numerical costs. 

We will now proceed further with assuming replica-symmetry (RS), viz. we require 
that permutation of spins within different replica groups leaves the order function ()12p 
invariant. In general, this amounts to assuming that the order function acquires e.g. the 
form 

P((t) = / dh W(h) rTTTTTT- (13) 

^ ' J ^ ' [2cosh(/?/i)]" ^ ^ 

since the sum over replica groups acts in the desired permutation-invariant way. In our 
case, where n can take any finite value, we find that for integer n values, significant 
simplifications occur if we rewrite replica-symmetry as 

n 
1=0 

for some density Q{x) which takes values only at a finite number of integer points and 
with the normalisation constraint 1 = Er=o {jjQi'^^ ~ "^^^ special case of ()14j) can 
also serve as a test of the accuracy of the more general expression of (fT^ . 

3.2. Thermodynamic quantities for general n 

Using the general-n RS assumption (jl3|) we may convert the self-consistent equation 
(fT^ into one for the effective field distribution W{h). To do so, first one expresses ^ 
in terms of W{h), so that / = extTw{h) f[W{h)] with: 



f[W{h)] = f dhdh'W{h)W{h') 

At) 



2/3 

X ^cosh"(^(p-2z/)) 
1 °° c^e~'^ I 

(■■■ 



1 + tanh(/5/i) tanh(/3/i') tanh(-(p - 2v)) 

c 

ncosh"(^(p-2z/0) 



X from now on we will use the proportionality symbol ~ to express distributions modulo their 
normalisation constant. 
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Figure 1. The distribution W{h) as obtained from equation H16(l for n = j^, c — 5, 
p = 7 and T — 0.05 (left picture, corresponding to a ferromagnetic pliasc) and T = 0.2 
(right picture, corresponding to a spin-glass phase). 



X 



[Y[dhiw{h,)]{ y: 

A=±l 



1=1 

with the averages 



n fl + Atanh(/3/i;)tanh(^(p-2z//)) 



.1=1 



fiy^ fp-i 



u=0 



V 



(15) 



Variation now of the above with respect to WiK) resuhs in 

oo 



fc=0 



ncosh"(^(p-2z/0) 



c 



1=1 



E n I 1 + Atanh(/3/iO tanh(^(p - 2z//)) 



A=±l / 



X (5 /i-^Eatanh[tanh(/3/i/)tanh(-(p-2i/;))) ) •■■) (16) 

In the special hmit n — this expression recovers El as it should. Typical profiles 
oiW(Ji) are shown in figure ^ 

Using the ansatze (jlOpiHj) one can also express the (condensed) retrieval and overlap 
order-parameters in terms of WiK): 



m = (^;^P^(o-)(Ti)t = fdh W{h) tanh(/5/i) 
q = (^P5(o-)aV2)^ = / tanh\(3h) 



(17) 



In figure El we present numerical solutions of equations ()17ll8p (left panel). 
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Figure 2. First-order R^SG transitions. Left picture: The order parameters m 
(solid) and q (dashed) for n — 0.1 and p/c — 8/5 using the general-n expressions 
(|17I18|I . Right picture: The order parameter m using the integer-n expressions H23[l 
with n = 2, c = 5 and p — 1,2, 3, 10 (from left to right). 



From equations ()16I17I18|) we can identify W{h) = 5{h) as the paramagnetic (P) 
solution. Using bifurcations analysis as in e.g. ^H] one can find possible second-order 
transitions from this state. Thus, making an expansion of p6p for small fields such that 
/ dhW{h)h^ = 0{e^) with < e ^ 1 we find that transitions to ferromagnetic (F: 
m,q ^ 0) and spin-glass regions (SG: m = 0, q ^ 0) are respectively given by 

1 = c ^cosh"(^(p - 2u) tanh(^(p - 2u))^ (19) 

1 = c ^cosh"(^(p - 2u) tanh2(^(p - 2z/))^ (20) 

In the limit n — > 0, when the evolution of connectivity variables is dominated the 
thermal noise T, the microscopic distribution of the Cij is uniform, and the calculation 
is equivalent to a replica theory with quenched randomness. In this limit the above 
transition lines then reduce indeed to the ones found in |15j. However, one must also 
keep in mind here that for increasing n, the phase diagram tends to be dominated by 
first-order transitions j2H]- In that case, fields can no longer be regarded as small close 
to the transition and a bifurcation analysis is not possible. One must then resort to 
strictly numerical methods evaluating directly all observables of interest. 

3. 3. Thermodynamic quantities for integer n 

We will now consider the special case where n is integer whereby replica symmetry 
implies the order function P(cr) can only have discrete arguments. Introducing the 
corresponding RS ansatz ()14j) and replacing the Kronecker 5-functions by their integral 
representations allows us to rewrite the self-consistent equation for P(cr) (fT^ in terms 
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of Q{£), namely 

Q{2s-n) r^expi^c(f2Qi2l-n)K{l,s,{p-2u))^ | (21) 
with the constants (in a particular integral representation) 

K{l,s,ip-2u)) = [2cosh[^ip-2uwj2J: h tanh^+'=[^(p - 2z/)] 

i=ofc=o W \ / ^ 

io 27r 

Equation can be solved numerically by iteration. In this representation, and also 
using the sublattice symmetric ansatz (fTUj) . we find that the (condensed) retrieval overlap 
order-parameter m = X^cr P{^)'^^ reads 



■ cos"(cu) tan^+*^(cj) cos {u{2l - n) + ^(3j + k)^ (22) 



VQ(2Z-n)2"/ — sin[(2/ -n)c.;]tan(c<;)cos"(cj) (23) 
T^n Jo 27r 



m ^ 

=0 

whereas, the spin glass order parameter q = J2a P{o-)o'^cr'^ is 

r2^ du 



q = y g(2Z - n)2" / — cos[(2Z - n)uj] tan^to) cos"(cj) (24) 
1=0 ^0 27r 

Finaly, the free energy in this representation is 

/ = - ^ ( E ( ^ ) Qi^' - ^) ^?(2^ - ^) ^(^ (2p - ^))^ 

+ i E ( ) ^(2^ - ^) logQ(2^ - ^) (25) 
Numerical solutions of equation (j^H)) are shown in figure |21 (right panel). 



3.4- Phase diagrams 

In figure El we present the phase diagram of our model for n = 0.1. It is drawn in the 
(a,T) plane where a = p/c. Direct evaluation of the observables m and q p6ll7ll8p 
shows that the P— >-R and P— *SG transitions occur at the lines predicted by (fTIHl and 
(1201). Generally, sufficiently small values of n lead to second-order P— >R and P^SG 
transitions, as was also the case for c — oo |2B]. As expected, the transition R— >SG 
is first-order and examples of m along this transition are shown in figure |21 Of special 
interest is the significant enlargament of the R phase for any n > 0. It physically implies 
that as soon as the connectivity variables {cy} can be 'mobile' the graph rearranges itself 
such that, for sufficiently low T, pattern recall can always be achieved. This effect is more 
apparent as the ratio n = [3 / (3 increases (see ffguresH}: the range in the parameter space 
where recall can be achieved increases, while the SG area shrinks. Numerical evaluation 
of equations ()23p24j) shows that large values of n (i.e. for n > 1) the P— i>R transition 
line becomes first-order above some value of the storage capacity a, which depends on 
c. In figure IH we have plotted phase diagrams for n = 2 and n = 3 for different values 
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Figure 3. Phase diagram for n = 0.1 and c = 5 in tlie space of T and a = p/c. Solid 
and dashed hnes represent second- and first-order transitions respectively while only 
markers correspond to physical points (where p is integer). Even small values of n 
lead to a significant enlargement of the retrieval phase (c.f. the phase diagram of [TS) 
where n — > 0). For sufficiently small values of n the P^R and P^SG transitions are 
second-order and the separating line is described from I|19I2U|I . 



of the connectivity. Compared to the extremely-diluted theory c — *• oo, we see that 
finite-connectivity leads to larger regions in parameter space where recall is possible (to 
illustrate this we have included the corresponding c — oo line in the left figure where 
n = 2). Physically, this can be understood on the basis of the number of connections 
per spin: for a system with finite c, there are enough vacancies for the connectivity 
variables to optimize the energetic contributions. However, for c oo, the number of 
vacancies for optimal configurations is less, creating an entropic countereffect. 



4. The fraction of misaligned spins 

Since our model aims to describe rearrangements of the connectivity matrix {cij} 
taylored to lead to optimal performance of pattern recall, we expect on physical grounds, 
that this will be in fact a result of a minimisation of the number of frustrated bonds in 
the system. To measure this effect let us introduce the joint distribution of spins and 
local fields 

p^i^, ^) = ^ E '^[^ - E T E ere^,] (26) 

With (|26|) we can now define the fraction of misaligned spins 

= (/^ d/i P^(l, h) + d/i P^(-l, K)\ (27) 
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Figure 4. Phase diagrams for n = 2 (left) and n = 3 (right) in the space of T and 
a — p/c. The transition hnes have been derived by direct evaluation of the observables 
TO (|23|l and q (|24|l and are of first order except for very small a. (also see figure 
Increasing the value of the parameter n leads to a shrinking of the SG phase. This is 
an immediate result of the minimisation of frustration in the graph. 



and the fraction of vanishing local fields 




The latter, in contrast with the c ^ oo regime, is expected not to vanish, since the 
distribution of local fields is a sum of delta peaks. In the limit c oo, this sum of 
delta peaks is traded for a continuous distribution, with a vanishing measure for the 
fraction of fields that are exactly zero. Expressions ^I7\ and fl28|) in combination will 
allow us to get an idea of the amount of frustration in our system. Note however 
that a small fraction of misaligned spins does not necessarily imply a small amount 
of frustration: for a large number of patterns the interactions J2fi^i^j have a large 
absolute value and therefore with a high prbability spins prefer to align to their local 
field. On the contrary, a vanishing number of frustrated bonds does imply a vanishing 
(p. If variations in n induce variations in 0, we expect this to be due to a change in the 
amount of frustration. 



4-.1. The joint distribution of spins and local fields 

To evaluate the joint distribution of spins and local fields we begin by inserting into the 
partition function (0} appropriate delta functions to define the local fields in the system 

i k ^ 

with {dhj} = na=i d/i". The delta function in the above expression introduces conjugate 
local fields in our equations. Therefore, we are required to consider the more general 
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(30) 



from which the joint distribution of spins and local fields follows simply by integrating 
out the conjugate fields h. Given and (IHUj). we can now take the trace over {cjj} 
in (j3)) and arrive at the free energy 



\<T(T' 



f = Extrip} <^ — (J2 / dhdh'dhdh' P^{ct, h, h) P^(cr, h, h) e" 



Kcr-h'+cr'-h) 



/ f . ih-ii+|hcr+c(X;cr'/dh'dii' p^,(cr,h,h) e-E('^*''+^ ^'')^,' 

+ ( log 2_] / dhdh e s 4 



(31) 



where the extremisation problem leads to the self consistent equation for P^{cr, h, h): 



P 



expUh-h + ^h-a- + c{y2j dh'dh'P^'(h', h', cr' 



-i(^-|')[h-cr'+h'.cr] 



(32) 



Let us now expand the exponential in the above right-hand side and integrate over the 
conjugate fields h: 



P^{a,h)= / dhP^(o-,h,h) 



~ e 



E 



e '^c'' 



k>0 



k\ 



k 

Y: J [ndh,dh,P^j(r,,h,,hO]e-^S.ea;T. 



\Ti-T,./ 1=1 



X6 



1=1 



(33) 



Using now the identity / dhdhe-*^ '^P^(<T, h, h) = e2« '^/dhdh P|(cr,h,h) we can 
write 



P^(o-,h)~et'^'^^ 



k=0 



k\ 



E WP^ir, 

Ti-Tfe/=1 



e^p{|:E(^^;)'T•T, 



X 5 



h-7E(^^^)^' 



(=1 



(34) 



where P^(cr) is the order function we defined in ((Zj). Equation represents the joint 
distribution of n spins and local fields. From here, we may find the single-replica joint 
distribution by integrating out n — 1 replicas. The result is 



P^(a,/.)~E 



fc=0 



k\ 



T,-Tkl=l 



n i 2 cosh 



xe^p{fE(^^^v^/}^ 



Q = 2 

1 fc 



3 ^ 
.^1=1 



c 



1=1 



(35) 
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This expression depends only on the order- function P^(cr). To proceed further one 
is now required to substitute either the general-n rephca- and sublattice-symmetric 
assumptions (jlHpifjj) . or, the corresponding integer-n ones ()14p2Hl . For general n we find 



that tracing over the spins {ti} results in lengthy expressions of questionable practical 
value. For integer n however, spins need not be necessarily traced: the fact that their 
dimensionality is prescribed to the well-defined integer value n allows us to proceed 
further and rewrite ()35|) in a numerically tractable form. Using the ansatz (fTUI) in the 
RHS of and the RS expression (fT^ we obtain 




2 cosh 



1=1 



X e 



1=1 



(36) 



with the abbreviated averages 




T s=0 



(37) 



(3^ 



(39) 



Once the density Q{i) has been obtained from ()2ip one can evaluate the distribution 
in the spirit of population dynamics [13]: one considers a population of triplets 
{i/, T,^}. One then selects a number k from a Poisson distribution of mean c and 
chooses I = 1, . . . ,k triplets {z//, r;, each according to the distributions ()37|39|38|1 . 
One then evaluates the location and weight of the local field h and thus estimates the 
probability P^{a, h) (with a and ^ fixed and modulo the normalisation coefficient). This 
process is repeated until convergence. 



4-.2. Numerical results 

In figure El we plot the fraction of misaligned spins ()27p as a function of the (spin) 
temperature T for c = 10 and a = 0.1, 0.5 and 1 respectively. As expected on physical 
grounds, we see that larger values of ra, i.e. lower temperatures T (or, lower timescales, 
or lower energies equivalently) associated to the slow wiring variables Cjj, alignment 
improves. This effect is due to the ordering of the q-,-, reducing the amount of frustration. 
In all cases we compare the results with those for c — > cxo (as derived in jSHj). Clearly, 
also for finite fixed c, (f) decreases as a function of a, which is in fact a result of the 
scaling choice of the interactions J2fj,^i^j/c- The expectation of their absolute value 
increases with p, as do the local fields, resulting in a high probability of alignment. We 
find that finite connectivity values of are always upper-bounded by those of c oo. 
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Figure 5. Fraction of misaligned spins 4> (|27|l for a — 0.1,0.5,1 (from left to right). 
In all figures c = 10 and results are compared to the extreme dilution limit c — > cx) 
(dotted lines), for n — 1,2, 3. 



For temperatures T sufficiently large, such that the system is in a paramagnetic state, 
the noisy dynamics of the spins dominates on the evolution of the graph and (f) becomes 
independent of the ratio n. 

The difference between finite and infinite connectivity, most clearly observed in the 
paramagnetic regime (large T), is found to be mainly due to a nonvanishing fraction ijj 
of sites that have a zero local field ()28|1 . This means that for such sites the energetic 
cost of either of the two possible orientations is identical. To understand the origin of 
the difference between finite-c and c ^ oo let us consider the quantity 

^ = <P+l^ (40) 

which represents the fraction of frustrated sites to which half of the 'indeterminate' 
spins have been added. This quantity is plotted in figure IHl for c = 14 and a = 0.5, and 
is compared to in the limit of c — >^ oo calculated in j2n]- We see that the finite- and 
infinite-c plots become almost identical apart from a small difference close to the R— 
transition. This hints that, as far as the fraction of misaligned spins is concerned, ip is 
the main discriminator between the two scaling regimes and that the choice of obtaining 
the c — » oo fraction via ()40|) is quite accurate. 

For large c, the quantity is expected to vanish. This is indeed supported by the 
results of figure[7l where ip is plotted as a function of c for T = 2, n = 2 and the memory 
loads a = 0.1, a = 0.5 and a = 1 respectively. Clearly, for small values of c the quantity 
ip can be rather large indeed, especially in the paramagnetic phase. It is also clear 
that the statistics for ip differ in the case of an odd versus an even number of patterns 
p, which we have plotted separately in the figures: for an odd number of patterns, 
there can be no zero- valued interactions S^^f^j*, whereas for even p this can be the 
case; hence the smaller values of ip for odd p. The magnitude of the difference in (p at 
T = 2 between finite connectivity and infinite connectivity in figure El appears to display 
a slight nonmonotonicity in a at ffist sight, but is consistent with the corresponding 
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Figure 6. Fraction of misaligned spins plus half of the fraction of spins with zero 
local field for c = 14, a = 0.5, n = 1, 2, 3, as compared to the case c ^ oo (dotted 
lines) . Differences between the two graphs are confined to temperature values close to 
the phase boundary. 
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Figure 7. Fraction of spins with zero field (|28() for a — 0.1,0.5,1 (from left to 
right). In all figures T = 2 and n = 2. Distinction has been made between even p 
(circles) and odd p (squares) . 



values for ip in figure Here the cases a 
whereas for a = 1 p is even. 

5. Simulations 



= 0.1 and a = 0.5 correspond to odd p, 



To perform simulation experiments one needs to construct an explicit dynamical process 
for the transition probabilities W[c'] c] between two graph configurations. To this end, 
one can easily set up Glauber-type probabilities iy[FjjC, c] that automatically obey 
detailed balance and which are functions of the energetic difference Hs{Fijc) — Hs{c), 
with Fij a 'switch' operator such that Fijf{cij) = /(I — q^) and Fijf{cke) = f{cki) if 
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Figure 8. Theory versus simulations: We plot the fraction i/j of spins with zero local 
field for a = 0.5, T = 2 and n = 2 (theory: squares, simulations: circles plus error 
bars) . Simulations are performed for a system of = 200 spins using the Glauber 
dynamics 141() . 



7^ {k, t). Explicit expressions on such processes can be derived equivalently to 
The result is 



tanh 



(2q,- - 1) 



log 



n 



log 



(41) 



2 \NJ 2 

The angular brackets denote a thermal average over the distribution Pf(cr) ~ 
exp[— /?iff(cr, c)] while spins are in equilibrium at the timescale of the graph dynamics. 
Note that this transition probability reduces to the one found in 26j for c — > cxd. 

The practical restrictions for 'coupled' types of simulations, where two nested 
dynamical processes occur, is that the total equilibration time can in reality be extremely 
large. So, for all practical purposes one is confined to small system sizes, which, 
unfortunately induce, in turn, large finite-size effects. Therefore, one has to accept 
that experiments will always suffer from a non-vanishing statistical error and can at 
most be satisfactory. 

With these practical limitations in mind, we will resort to compare our theory for 
the fraction ip of zero local fields in the paramagnetic regime for n = 2 and T = 2, and 
for small c. In this way we exploit the fact that in the paramagnetic regime equilibration 
times are relatively short. In figure |H1 we have plotted ip for the 5 first possible values of 
c at a = 0.5, with T = 2 and n = 2. Even though the system size is still rather modest 
(we used N = 200), the results clearly support the theory. 



6. Conclusions 



It is a relatively new insight that a wide range of physical complex systems (viewed as 
random graphs) evolve towards specific structures and are, surprisingly, characterised by 
universal features (e.g. scale-free degree distributions). Preferential attachment models 
describing network growth, such as the Barabasi- Albert algorithm j2], provide possible 
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explanations for the occurrence of certain network-architectures. In the present paper 
we have presented a simple solvable model describing evolving random graphs of a fixed 
number of nodes, with finite connectivity, in which the analytic hurdles imposed by the 
explicit dynamics are overcome by taking the adiabatic limit: we consider connectivities 
which evolve much slower than nodes. This allows us to study directly thermodynamic 
properties of graphs at equilibrium. 

In particular, we have focused our attention on Hopfield neural network models on 
random graphs. By a suitable choice of chemical potential, the average connectivity 
per neuron is forced to a finite regime, as opposed to a previous study [21] in which it 
was infinite. The resuhing theory is, traditionally (see jlHlliniEniEIlESEniEl), a 
finite dimensional replica theory, in which the replica dimension represents the ratio of 
temperatures between the fast and the slow system. 

The consequences of the finite connectivity regime, as compared to the regime of so 
called extreme dilution, are drastic at the mathematical level, since a finite set of order 
parameters is traded for an infinite dimensional order parameter function. The case of 
integer replica dimension, however, is an exception to the latter statement, simplifying 
the mathematics, so that observables can be calculated accurately. In this paper we 
have presented phase diagrams and observables for both integer replica dimension and 
non-integer replica dimension. For integer replica dimension, we were able to calculate 
the fraction of misaligned spins, as well as the fraction of vanishing fields. The latter 
quantity, which vanishes in the case of infinite connectivity, is the most important 
discriminator between the parameter regimes of finite connectivity and extreme dilution. 
Qualitatively, the frustration effects are minimised as the temperature of the slow system 
decreases, enhancing the retrieval state of a condensed pattern. Moreover, for larger n, 
phase transition lines are mainly first order, and the retrieval phase is dramatically 
enlarged in comparison with the case n = 0. For any nonzero n there is no critical 
storage capacity beyond which retrieval is not possible. In other words, as soon as 
the statistics for the graph realisations becomes nonuniform, the graph in principle can 
organise itself in favour of the retrieval of a pattern, regardless of the number of patterns 
present in the Hopfield interactions (note that this statement is in principle true for an 
arbitrarily large but finite p, due to the scaling regime of finite connectivity). Our 
theoretical findings are supported by simulation results, which, due to the difficulties 
of nested equilibrations, we have restricted to a simple (paramagnetic) region in the 
phase diagram. All results obtained in this paper are replica symmetric approximations. 
However, as in the case of infinite connectivity 26j, we expect replica symmetry breaking 
to occur only at sufficiently low values of n i.e. below n = 1, and small values of T or 
large values of a. 

Several extensions of this work are possible: in a similar fashion one may also 
consider evolution models of random poissonian graphs to e.g. scale-free ones. In the 
context of the present paper this amounts to appropriate modifications of our chemical 
potential while the remaining theory would remain largely the same. 

Furthermore, one may investigate the local stability of non-condensed pattern 
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retrieval states once the Cij are equilibrated in favour of one particular pattern, as 
was done in To that end, one needs to overcome the problem of taking the replica 
limit n — in only one of two spin systems {cTj} and {tj}, described by a coupled 
order function -P^(cr, r). So far, we have not been able to solve this problem. 

Acknowledgment 

The authors are indebted to A.C.C. Coolen, J.P.L. Hatchett, T. Nikoletopoulos 
and I. Perez-Castillo for illuminating discussions on finite connectivity issues. BW 
acknowledges financial support from Stichting FOM (Fundamenteel Onderzoek der 
Materie) in the Netherlands and NS from the Ministerio de Educacion, Cultura y 
Deporte (Spain, grant SB2002-0107) and the ESF SPHINX program. 



References 



Dorogovtsev S N and Mendes J F F 2002 Adv Phys 51 1079 
Albert R and Barabasi A L 2002 Rev Mod Phys 74 47 
Viana L and Bray A J 1985 J. Phys. C 18 3037 
Kanter I and Sompolinsky H 1987 Phys Rev Lett 58 164 
Mezard M and Parisi G 1987 Europhys Lett 3 1067 
Wong K Y and Sherrington D 1987 J Phys A 20 L793 
Kabashima Y and Saad D 2003 J Phys A 37 R1-R43 

Skantzos N S, van Mourik J, Saad D and Kabashima Y 2003 J Phys A 36 11131 
Kabashima Y, Murayama T and Saad D 2000 Phys Rev Lett 84 1355 
Skantzos N S, Saad D and Kabashima Y 2003 Phys Rev E 68 056125 
Cocco S and Monasson R 2001 Phys Rev Lett 86 1654 
Monasson R and Zeechina R 1997 Phys Rev E 56 1357 
Mezard M and Parisi G 2001 Eur Phys J B 20 217 

Nikoletopoulos T, Coolen A C C, Perez Castillo I, Skantzos N S, Hatchett J P L and Wemmenhove 

B 2004, submitted to J Phys A, cond-mat/0402504 
Wemmenhove B and Coolen A C C 2003 J Phys ^ 36 9617 
Perez-Castillo I and Skantzos N 2003 cond-mat/0309655 

Perez-Castillo I, Wemmenhove B, Hatchett J P L, Coolen A C C, Skantzos N S and Nikoletopoulos 

T 2003, submitted to J Phys A, cond-mat/0404018 
Penney R W, Coolen A C C and Sherrington D 1993 / Phys A 26 3681 
Coolen A C C, Penney R W and Sherrington D 1993 Phys Rev B 48, 16 116 
Jongen G, BoUe D and Coolen A C C 1998 J Phys A 31 L737 

Jongen G, AnemuUer J, BoUe D, Coolen A C C and Perez- Vicente C 2001 J Phys A 34 3957 
Uezu T and Coolen A C C 2002 J Phys A 35 2761 
Feldman D E and Dotsenko V S 1994 J Phys A 27 4401 
Dotsenko V, Franz S and Mezard M 1994 / Phys A 27 2351 
van Mourik J and Coolen A C C 2001 / Phys ^ 34 LI 11 



Wemmenhove B, Skantzos N S and Coolen A C C 2004, submitted to J Phys A, cond-mat/0403615 
van Hemmen J L and Kiihn R 1986 Phys Rev Lett 57 913 
Sherrington D 1980 J Phys A 13 637 



0.15 



0.125 



0.1 



0.075 



0.05 



0.025 







